On making causal claims: A review and recommendations

Social scientists often estimate models from correlational data, where the independent variable has not been exogenouslymanipulated; they alsomake implicit or explicit causal claims based on thesemodels.When can these claimsbemade?Weanswer this questionby first discussing design and estimation conditionsunderwhichmodel estimates can be interpreted, using the randomized experiment as the gold standard. We show how endogeneity –which includes omitted variables, omitted selection, simultaneity, common-method variance, and measurement error – renders estimates causally uninterpretable. Second, we present methods that allow researchers to test causal claims in situations where randomization is not possible or when causal interpretation could be confounded; these methods include fixed-effects panel, sample selection, instrumental variable, regression discontinuity, and difference-in-differences models. Third, we take stock of themethodological rigorwithwhich causal claims are beingmade in a social sciences discipline by reviewing a representative sample of 110 articles on leadership published in the previous 10 years in top-tier journals. Our key finding is that researchers fail to address at least 66% and up to 90% of design and estimation conditions that make causal claims invalid. We conclude by offering 10 suggestions on how to improve non-experimental research. © 2010 Elsevier Inc. All rights reserved.